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ABSTRACT 


The  goal  of  this  research  has  been  to  determine  those  areas 
which  management  should  consider  as  top  priorities  for  improv- 
ing the  success  of  the  information  systems  function.  In  order 
to  achieve  this,  an  integrated  definition  of  I/S  success  was 
developed  and  used  to  evaluate  the  I/S  function  in  nineteen 
industrial  firms.  Using  the  results  of  these  assessments, 
areas  that  will  maximize  the  improvement  in  overall  I/S  success 
are  identified  as  the  top  priorities  for  I/S. 

The  paper  begins  with  an  assessment  of  the  current  situation; 
first,  how  severe  is  the  problem  with  I/S  success  and  second, 
what  are  the  relative  strengths  and  weaknesses  of  typical  I/S 
departments.  A  definition  of  I/S  success  is  developed  which 
includes  26  criteria  covering  the  range  of  I/S  activities.  The 
measurement  of  success,  based  on  the  opinions  of  both  user  and 
I/S  managers,  includes  a  rating  of  the  relative  importance  of 
each  criterion,  as  well  as  a  rating  of  its  performance.  The 
results  indicate  that  there  is  in  fact  a  problem  in  overall  I/S 
success,  especially  in  effectiveness-oriented  criteria. 

The  underlying  cause  for  the  low  rating  of  overall  success  is 
then  explored.  In  searching  for  this  basic  cause,  agreement  is 
found  between  I/S  and  user  managers  that  is  both  surprising  and 
encouraging.  Not  only  do  they  agree  in  their  views  of  I/S  suc- 
cess, but  their  agreement  extends  to  the  components  of  success 
as  well:  the  dimensions  of  importance  and  performance.  Con- 
trary to  popular  opinion,  the  cause  of  low  success  is  not  to  be 
found  in  disagreements  betv/een  user  and  I/S  managers. 

Instead,  the  cause  of  mediocre  overall  success  ratings  and  low 
success  on  specific  criteria  is  to  be  found  in  the  undeniable 
lack  of  a  relationship  between  what  is  important  for  I/S  and 
what  I/S  performs  well.  This  lack  of  a  managerially  justifia- 
ble relationship  between  importance  and  performance  is  seen 
not  only  by  users  but  by  I/S  professionals  as  well. 
Apparently,  I/S  management  has  not  been  able  to  effectively 
allocate  resources  so  that  the  relative  performance  of  the  26 
success  criteria  matches  their  respective  relative 
importances . 

The  diagnosis  of  the  gap  between  importance  and  performance  as 
the  cause  of  low  success  provides  the  basis  for  establishing 
the  top  priorities  for  I/S.  To  maximize  improvement  in  overall 
I/S  sucess,  the  I/S  department  must  direct  its  improvement 
efforts  to  those  criteria  where  relative  importance  is  signif- 
icantly greater  than  relative  performance.  These  top  priority 
criteria  are  then  classified  in  order  to  facilitate  integrated 
planning  for  both  structural  and  task-oriented  improvement 
efforts. 

This  process  of  defining,  measuring,  and  diagnosing  I/S  suc- 
cess is  a  practical,  effective  method  for  identifying  top  pri- 
orities for  I/S  strategic  planning  and  resource  allocation. 
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INTRODUCTION 


Computer  systems  have  become  critical  to  the  daily  operation  of 
most  large  companies.  In  fact  for  many  corporations, 
computer-based  information  systems  have  become  key  to 
competitiveness  in  the  marketplace.  In  addition  to  supporting 
the  routine  operational  functions,  computer-based  information 
systems  are  being  used  more  and  more  to  support  the  information 
needs  of  managers. 

In  a  continually  changing  technical  environment,  information 
systems  are  also  changing  the  nature  of  their  relationship  with 
the  rest  of  the  company.  It  is  thus  critical  that  the  informa- 
tion systems  (I/S)  function  be  well-managed.  The  I/S  depart- 
ment must  allocate  its  resources,  particularly  its  scarce 
management  and  personnel  resources,  in  order  to  best  meet  the 
company's  business  needs.  For  I/S  planning  to  be  relevant  and 
effectively  implemented,  senior  and  middle  level  user  managers 
must  be  involved  in  assessing  success,  setting  priorities,  and 
allocating  resources.  The  company's  top  priorities  for  the 
information  systems  function  must  be  understood  and  endorsed 
by  both  I/S  and  user  management. 

However,  any  process  of  setting  priorites  is  frequently  under- 
mined by  the  need  for  short-term,  if  not  immediate,  results. 
Often  priorities  are  assigned  on  the  basis  of  the  "squeaky 
wheel"  phenomenon  or  by  power  politics,  not  on  the  basis  of  the 
most  important  underlying  needs. 

A  company  should  be  able  to  assess  the  success  of  its  I/S 
department  and  to  develop  priorities  for  action  that  will 
increase  overall  success.  The  relative  strengths  and  weak- 
nesses of  all  I/S  activities  should  be  considered.  Only  by 
including  the  breadth  of  I/S  activities  can  trade-offs  be  made 
when  setting  priorities  for  I/S  management. 

To  be  effective  in  improving  I/S  success,  management  must  be 
able  to  diagnose  success,  differentiate  areas  of  relative 
strengths  and  weaknesses,  and  prioritize  resource  allocation 
on  the  basis  of  the  greatest  contribution  to  overall  success. 
This  paper  presents  such  a  diagnosis,  differentiation,  and 
prioritization,  based  on  a  survey  of  a  thousand  I/S  and  user 
managers  from  nineteen  industrial  firms. 

First,  this  paper  develops  a  conceptual  framework  that  pro- 
vides an  integrated  definition  of  I/S  success,  across  the 
breadth  of  I/S  activity.  Secondly,  it  diagnoses  success  and 
uses  the  discovered  cause  of  low  success  as  the  basis  for 
developing  the  top  priorities  for  the  I/S  function.   The 


The  authors  wish  to  gratefully  acknowledge  the  assistance  of 
Charles  Dickinson  in  preparing  this  paper. 
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results  are  encouraging.  Not  only  does  there  seem  to  be  strong 
agreement  between  user  managers  and  I/£  managers  regarding  the 
current  level  of  I/S  success,  but  there  also  is  good  agreement 
on  the  top  priorities  for  its  improvement. 


THE  USER  NEEDS  SURVEY  SAMPLE 


The  thousand  managers  whose  responses  are  being  used  to  assess 
I/S  success  were  surveyed  as  part  of  a  large  research  pro ject, 
the  User  Needs  Survey.  The  project  is  being  conducted  at 
M.I.T.'s  Center  for  Information  Systems  Research  (CISR),  with 
Dr.  Robert  M.  Alloway  as  Principal  Investigator.^  The  purpose 
of  the  User  Needs  Survey  is  to  recommend  improved  information 
systems  policies  and  procedures.  The  study  was  motivated  by 
the  desire  to  better  understand  the  managers  who  are  end  users 
of  computer-based  systems  and  the  issues  involved  in  fulfill- 
ing their  information  needs. 

Nineteen  companies  from  the  industrial  sector  participated  in 
the  project.  As  summarized  in  Figure  1,  they  differed  in 
industrial  classification,  size  ( as  measured  by  revenues ) ,  and 
I/S  department  budget  in  relation  to  total  revenues.  The  com- 
panies also  differed  in  corporate  structure,  in  the  size  and 
structure  of  their  I/S  departments,  and  in  the  reporting 
relationship  of  the  I/S  department  within  the  company. 

In  each  company,  the  research  team  selected  managers  from  the 
I/S,  Finance  and  Manufacturing  departments  to  complete  a  com- 
prehensive questionnaire  which  addressed  a  range  of  issues 
related  to  the  I/S  function. 

The  respondents  were  chosen  to  be  a  representative  sample  of 
both  the  managerial  levels  and  the  job  functions  in  each  of  the 
three  departments.  Figure  2  shows  that  of  the  1058  managers 
surveyed,  just  over  forty  percent  were  I/S  professionals  and 
just  under  sixty  percent  were  users.  Level  1  respondents  are 
the  heads  of  their  respective  departments,  and  level  4  are 
first  line  managers. 

The  User  Needs  Survey  methodology  was  carefully  designed  and 
well  implemented  to  ensure  relevant,  .ligh  quality  data.  The 
survey  has  provided  a  foundation  for  the  analysis  of  the  compo- 
nents of  I/S  success  and  the  development  of  a  set  of  top  priori- 
ties for  improving  I/S  success  in  the  future. 


For  a  description  of  this  project,  see  Alloway,  R.M.,  C.V. 
Bullen,  and  J. A.  Quillard,  "Overview  of  the  User  Needs  Sur- 
vey Research  Project",  CISR  Working  Paper  #73 ,  M.I.T.,  June 
1981. 


Introduction 


FIGURE  1. 


PROFILE  OF  FIRMS  SURVEYED 


INDUSTRY  CLASSIFICATION  FOR  THE  19  FIRMS 


o  Paper,  fiber  and  wood  products 

o  Rubber,  plastics  products 

o  Chemicals 

o  Aerospace 

o  Communications 

o  Food  processing 

o  Tobacco  products 

o  Textiles 

o  Motor  vehicles 

o  Office  equipment 

o  Measuring,  analyzing,  and  control  equipment 

o  Electronics 


SIZE  OF  FIRMS 


Revenue  Range 


over 

$  10 

billion 

$  5 

billion  - 

$  10 

billion 

$   1 

billion  - 

$   5 

billion 

$500 

million  - 

$   1 

billion 

$100 

million  - 

$500 

million 

under  $100  million 


Number  of 
Sample  Firms 


Firms'  Parent 
Organizations 

4 
3 
6 
2 
2 
2 

19 


I/S  BUDGET  AS  A  PERCENT  OF  FIRMS'  REVENUES 


Percentage  Range 


Number  of  Firms 


2%  -  3% 
1%  -  2% 
0.5%  -  1% 
0.25%  -  0.5% 
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FIGURE   2. 


NUMBER  OF  RESPONDENTS 


*LEVELS 

i/s 

MFG 

FINANCE 

1 

26 

18 

18 

2 

37 

40 

35 

3 

96 

115 

84 

4 

289 

159 

141 

TOTALS 

448 

332 

278 

TOTALS 

62  (  5.9%) 
112  (10.6%) 
295  (27.9%) 
589    (55.6%) 

1058    (   100%) 


(42.3%)    (31.4%)  (26.3%) 


*  Level    1   =  Department   head 

Level   4  =  First-line  manager;   average   salary:      $25,000 
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WHERE  ARE  THE  PROBLEMS  IN  l/S  SUCCESS? 


In  order  to  determine  whether  or  not  there  is  a  problem  with  I/S 
success,  and  where  those  problems  might  be  located,  we  need 
first  to  define  success  for  the  information  systems  function. 
Unfortunately,  defining  success  for  I/S  is  akin  to  hitting  a 
moving  target.  There  has  been  dramatic  growth  in  demand  for  new 
systems  applications,  with  an  associated  impact  on  the  atti- 
tudes and  opinions  of  user  and  I/S  managers  alike.  ^  This 
explosion  of  I/S  activity  has  resulted  from  the  simultaneous 
interaction  of  many  factors,  including  continued  dramatic 
improvements  in  hardware  technology,  increased  dependence  of 
modern  corporations  on  information  processing,  more  corporate 
resources  dedicated  to  the  I/S  function,  and  a  growing  computer 
literacy  among  users. 


DEFINING  I/S  SUCCESS 


A  definition  of  I/S  success  must  take  into  account  the  changing 
relationship  between  users  and  the  I/S  function  in  corporate 
organizational  environments  today.  It  must  incorporate  the 
breadth  and  diversity  of  factors,  such  as  operations,  mainte- 
nance, systems  development,  long  range  planning,  and  user 
interface,  which  constitute  the  scope  of  I/S  responsibility  in 
such  settings. 

Figure  3  provides  the  set  of  criteria  developed  for  the  User 
Needs  Survey  which  can  be  used  as  a  basis  for  defining  I/S  suc- 
cess. It  is,  in  some  respects,  an  arbitrary  list  of  criteria 
which  we  believe  to  be  relevant  to  all  I/S  departments.  The  set 
of  criteria  is  not  intended  to  be  exhaustive.  Instead  it  is 
designed  to  be  a  high  level  aggregation  of  activities  repres- 
enting the  breadth  and  diversity  of  I/S  functions. 

In  particular,  the  User  Needs  Survey  success  criteria  repre- 
sent the  three  different  categories  of  planning  and  control 
activities  which  must  take  place  in  a  growing,  adaptive  organ- 
ization. These  planning  and  control  activities  have  been  iden- 
tified by  Robert  Anthony  as  strategic,  tactical  and 
operational.^  Each  of  these  categories  is  represented  by  many 


Alloway,  R.M. ,  "User  Managers  Systems  Needs",  CISR  Working 
Paper  #56,  M.I.T.,  May  1980.  This  paper  differentiates  the 
demand  for  information  systems  among  several  types  and  also 
discusses  the  level  of  demand  for  each  system  type. 
Anthony ,  R . N . ,  Planning  and  Control  Systems:  A  Framework 
for  Analysis,  Harvard  University  Graduate  School  of  Busi- 
ness Administration,  Boston,  1955. 
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FIGURE  3  .         CRITERIA  FOR  DEFINING  I/S  SUCCESS 


A.  COMMUNICATION  WITH  MANAGERIAL  USERS 

B.  EFFICIENCY  OF  HARDWARE  UTILIZATION 

C.  HARDWARE  AND  SYSTEMS  DOWNTIME 

D.  TRAINING  PROGRAMS  FOR  USERS  IN  GENERAL  I/S  CAPABILITIES 

E.  DATA  SECURITY  AND  PRIVACY 

F.  QUALITY  OF  I/S  SYSTEMS  ANALYSTS 

G.  THE  ATTITUDE  OF  I/S  PERSONNEL  TOWARD  USERS 
H.  TECHNICAL  COMPETENCE  OF  THE  l/S  STAFF 

I.  THE  NEW  SYSTEM  REQUEST  BACKLOG 

J.  DEVELOPING  MORE  MONITOR  SYSTEMS 

K.  DEVELOPING  MORE  EXCEPTION  SYSTEMS 

L.  DEVELOPING  MORE  INQUIRY  SYSTEMS 

M.  DEVELOPING  MORE  ANALYSIS  SYSTEMS 

N.   INVOLVEMENT  OF  SENIOR  USER  MANAGERS  IN  I/S  POLICY  FORMULATION 
AND  EVALUATION 

0.   RESPONSIVENESS  TO  USER  NEEDS 

P.   I/S  STRATEGIC  PLANNING  AND  ALLOCATION  OF  RESOURCES  TO  KEY 

BUSINESS  AREAS 
Q.   INCREASING  THE  PROPORTION  OF  I/S  3FF0RT  EXPENDED  IN  CREATING 

NEW  SYSTEMS 
R.   TECHNICAL  SOPHISTICATION  OF  NEW  SYSTEMS 

S.   IMPROVING  NEW  SYSTEMS  DEVELOPMENT  (TIME,  COST,  QUALITY, 

DISRUPTIONS) 
T.   USER  ORIENTED  SYSTEMS  ANALYSTS  WHO  KNOW  USER  OPERATIONS 
U.   I/S  SUPPORT  FOR  USERS  IN  PREPARING  PROPOSALS  FOR  NEW  SYSTEMS 
V.   APPROPRIATE  I/S  BUDGET  SIZE  OR  GROWTH  RATE 
W.   AVAILABILITY  AND  TIMELINESS  OF  REPORT  DELIVERY  TO  USERS 

X.   RUTOJING  CURRENT  SYSTEMS  (COSTS,  EASE  OF  USE,  DOCUMENTATION, 

MAINTENANCE) 
Y.   REPORT  CONTENTS  (RELEVANCE,  CURRENTNESS,  FLEXIBILITY,  ACCURACY) 
Z.   I/S  PROFITABILITY  (FROM  CHARGEOUTS  FOR  SERVICES) 
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criteria  in  the  User  Needs  Survey  definition  of  success,  as  the 
following  examples  show: 

Strategic  activities 

N.  Involvement  of  senior  user  managers  in  I/S  policy 

formulation  and  evaluation 
O.  Responsiveness  to  user  needs 
P.  I/S  strategic  planning  and  allocation  of  resources 

to  key  business  areas 

Tactical  activities 

H.  Technical  competence  of  the  I/S  staff 
S.  Improving  new  systems  development 
U.  I/S  support  for  users  in  preparing  proposals  for 
new  systems 

Operational  activities 

B.  Efficiency  of  hardware  utilization 

C.  Hardware  and  sysrems  downtime 

W.  Availability  and  timeliness  of  report  delivery 
to  users 

In  addition,  the  list  of  criteria  can  be  related  to  a  variety  of 
technical,  operational,  organizational,  personnel,  and  plan- 
ning concerns. 

Beyond  this  balance  by  activity  and  topic,  the  25  success  cri- 
teria effectively  represent  the  interests  and  needs  of  users  as 
well  as  I/S  professionals.  The  set  of  criteria  is  not  exclu- 
sively concerned  with  traditional  technical  "DP"  requirements. 
Instead,  it  is  explicitly  directed  towards  user  concerns  as 
well,  for  example,  training  programs  for  users  (D), 
user-oriented  systems  analysts  (T),  and  communication  with 
managerial  users  (A)  . 

The  set  of  26  criteria  therefore  provides  us  with  a  broad  view 
of  success  for  the  I/S  function  in  its  current  dynamic  environ- 
ment. It  provides  us  with  a  balanced  and  easily  understood 
basis  for  assessing  the  I/S  function  in  organizations  like  the 
nineteen  firms  studied  in  the  User  Needs  Survey.  As  an  inte- 
grated definition  of  success,  it  enables  not  only  an  overall 
measurement  but  also  an  assessment  of  relative  success  for  each 
criterion.  But  we  still  need  a  method  which  allows  us  to  actu- 
ally assess  --  or  measure  --  I/S  success  in  the  context  of  a 
particular  firm  or  organization. 
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MEASURING  l/S  SUCCESS 


Interviews  with  user  managers  have  consistently  highlighted 
the  need  to  account  for  two  different  dimensions  when  actually 
assessing  the  success  of  the  I/S  function  in  a  particular 
organization.  The  first  is  the  relative  importance  of  a  par- 
ticular activity  (or,  in  this  case,  a  particular  success  crite- 
rion) .  The  second  is  the  actual  performance  on  that  activity 
or  criterion.  Measuring  overall  I/S  success,  therefore,  real- 
ly requires  that  we  assess  the  relative  importance  of  each  of 
our  26  success  criteria  as  well  as  I/S's  actual  performance  on 
these  same  criteria. 

This  distinction  between  importance  and  performance  suggests 
that  to  be  rated  as  highly  successful,  an  I/S  criterion  must 
not  only  have  good  performance,  but  must  also  be  acknowledged 
by  I/S  and  user  managers  to  be  a  relatively  important  one  as 
well.  Obviously,  low  performance  on  a  very  important  I/S  cri- 
terion should  not  be  considered  as  very  successful. 
Conversely,  a  high  performance  rating  for  a  criterion  of  rela- 
tively low  importance  can  not  be  considered  to  be  highly 
successful  either. 

Measuring  I/S  success,  therefore,  is  not  simply  a  task  of  meas- 
uring performance.  It  requires  that  we  develop  a  measure  of 
performance  that  is  affected  (or  weighted)  by  a  similar  measure 
of  relative  importance.  In  order  to  allow  for  ratings  on  both 
dimensions,  the  User  Needs  Survey  established  rating  scales 
which  allowed  I/S  and  user  managers  to  rate  both  the  perform- 
ance and  the  importance  of  all  26  I/S  success  criteria.  These 
scales,  both  of  which  have  a  range  extending  from  a  low  rating 
of  "1"  to  a  high  rating  of  "7",  are  illustrated  in  Figure  4. 

Using  this  methodology  to  measure  I/S  success,  the  User  Needs 
Survey  asked  a  thousand  user  and  I/S  managers  in  nineteen  dif- 
ferent firms  to  rate  the  performance  and  the  relative  impor- 
tance of  all  26  success  criteria.  These  two  ratings  were  then 
combined  through  the  weighting  algorithm  shown  in  Figure  4  to 
arrive  at  an  overall  success  rating.  When  aggregated  across 
all  criteria,  we  were  able  to  develop  an  overall  success  rating 
for  the  I/S  department  in  each  firm.  In  addition,  we  were  able 
to  develop  success  ratings  for  each  of  the  26  criteria  for  the 
entire  sample  of  a  thousand  managers. 


OVERALL  I/S  SUCCESS 


What  did  the  success  ratings  of  user  and  I/S  managers  indicate 
about  the  success  of  the  I/S  function  in  the  nineteen  firms? 
The  combined  success  ratings  suggest  that  there  is,  indeed,  a 
problem  with  I/S  success.  The  bottom  of  Figure  5  shows  that 
the  overall  success  rating  for  all  nineteen  firms  is  only  4.28, 
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FIGURE  4. 
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FIGURE  5.     AVERAGE  SUCCESS  RATINGS  BY  COMPANY 


COMPANY        I/S  MANAGERS        USER  MANAGERS        ALL  MANAGERS 


1  5.00 
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13  4.39 
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16  4.47 
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AVERAGE  4.43  4.19  4.28 
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approximately  midway  between  "Inadequate"  and  "Good"  on  the 
scale. 

It  is  not  surprising  that  average  overall  success  for  nineteen 
firms  is  in  the  middle  range  of  the  scale.  Ratings  at  the  two 
extremes  of  the  scale  are  both  unlikely  possibilities  for  any 
extended  period  of  time.  Ratings  at  the  "Inadequate"  level  of 
success,  for  example,  would  not  continue  for  long  because  the 
organization  would  probably  experience  major  upheavals  -- 
staff  fired  or  moved,  organizational  units  reorganized,  new 
leadership  brought  in  from  the  outside  --  before  matters  dete- 
riorated further  to  the  "Very  Poor"  level.  An  "Excellent" 
rating  of  "7"  is  also  unrealistic.  This  is  not  only  because  of 
the  growth,  change,  and  complexity  of  the  current  I/S  environ- 
ment, but  also  because  managers'  expectations  of  what 
constitutes  excellence  (in  this  or  any  other  function)  change 
as  success  improves  over  time. 

The  range  of  overall  success  ratings  for  the  nineteen  firms 
confirms  this  understanding  of  the  success  scale's  relevant 
range.  The  highest  users'  rating  for  a  firm  was  4.86  (ap- 
proaching "Good"),  while  the  lowest  users'  rating  was  3.38  (ap- 
proaching "Inadequate"). 

However,  it  does  not  seem  unrealistic  to  expect  that  overall 
success  be  rated  at  "5",  or  "Good",  on  the  scale.  Instead,  it 
has  been  placed  by  the  thousand  user  and  I/S  managers  nearer 
"4",  the  midpoint  of  the  scale.  This  indicates  rather  clearly 
that  industry-wide,  the  I/S  function  does  have  a  problem  with 
overall  success.  Not  surprisingly,  I/S  managers  rated  overall 
I/S  success  somewhat  more  highly  (4.43)  than  did  user  managers 
(4.19).  Even  so,  the  I/S  managers'  rating  is  not  a  strong 
endorsement  of  their  own  success,  as  it  falls  well  below  a 
"Good"  rating. 


RELATIVE  STRENGTHS  AND  WEAKNESSES  IN  I/S  SUCCESS 


The  average  success  ratings  of  the  26  individual  criteria  are 
organized  in  descending  sequence  in  Figure  6.  They  are  dis- 
tributed across  a  range  that  extends  from  a  high  of  5.01  (cri- 
terion J,  developing  more  Monitor  systems)  to  a  low  of  3.27 
(criterion  D,  training  programs  for  users).  Managers  clearly 
recognize  that  certain  functions  and  activities  within  the  I/S 
function,  as  represented  by  the  26  success  criteria,  are  rela- 
tively more  successful  than  others. 

If  the  success  ratings  in  Figure  6  are  partitioned  approxi- 
mately at  the  midpoint  and  quartile  levels,  an  interesting  pat- 
tern emerges.  The  criteria  in  each  of  the  four  groups  are 
generally  related  to  each  other.  The  first  group,  those  which 
received  the  highest  success  ratings,  reflects  the  traditional 
technical  activities  that  data  processing  has  engaged  in  over 
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the  years.  It  is  not  surprising  that  these  criteria  are  seen  to 
be  the  most  successful  of  the  entire  set.  The  criteria  in  the 
second  group  are  associated  with  the  operations  of  hardware  and 
existing  application  systems.  Again,  given  the  accumulated 
experience  and  attention  I/S  has  focused  in  this  area,  it  is 
not  too  surprising  that  this  group  is  rated  by  I/S  and  user  man- 
agers as  an  area  of  relative  strength  --  though  certainly, 
there  is  room  for  improvement. 

Below  average  success  ratings  comprise  the  third  group.  It 
includes  criteria  which  are  associated  with  I/S  management, 
and  the  relatively  lower  ratings  suggest  comparative  weakness. 
Finally,  the  last  group  of  ratings  involves  criteria  that  are 
associated  with  the  development  of  managerial  support  systems. 

A  number  of  observations  are  suggested  by  this  pattern  of  the 
success  ratings  for  the  26  criteria.  First,  the  pattern  as  a 
whole  makes  sense  in  suggesting  that  the  I/S  departments  m  the 
nineteen  firms  are  most  successful  at  such  traditional  techni- 
cal activities  as  developing  monitoring  systems  (J),  maintain- 
ing the  technical  competence  of  the  I/S  staff  (H) ,  and 
establishing  technically  sophisticated  systems  (E).  The  pat- 
tern clearly  indicates  that  the  I/S  function  is  least 
successful  in  activities  that  require  the  development  of  flex- 
ible, user-oriented  systems,  such  as  inquiry  and  analysis 
systems  (L,M),  as  well  as  the  concommitant  user-interface 
capabilities  such  as  user-oriented  systems  analysts  (T), 
involving  senior  user  managers  in  I/S  policy  formation  (N)  ,  and 
training  programs  for  users  (D) .  Areas  of  relative  strength 
occur  in  efficiency-oriented  criteria  whereas  criteria  of  rel- 
ative weakness  are  associated  with  effectiveness. 

Secondly,  the  pattern  suggests  that  there  may  be  an  experience 
or  learning  curve  at  work  in  the  I/S  function,  whereby  capabil- 
ities and  procedures  improve  with  management  attention, 
volume,  and  increased  specialization.  Operations,  for 
example,  used  to  be  a  problem  in  the  data  processing  field. 
Hardware  was  inefficiently  utilized,  system  downtime  was  often 
a  problem,  and  reports  suffered  from  unsatisfactory  timeliness 
and  availability.  However,  the  pattern  of  success  ratings  dem- 
onstrates that  this  area  is  currently  one  of  relative  strength, 
at  least  as  compared  with  I/S  management  and  the  development  of 
more  managerial  support  systems.  The  pattern  indicates  that 
these  latter  areas,  and  particularly  that  of  managerial  sup- 
port systems,  will  require  time,  resources,  and  especially 
management  attention  before  we  learn  how  to  implement  proc- 
esses which  ensure  success.'' 


*  See  Gibson,  C.F.,  and  Nolan,  R.L.,  "Managing  the  Four 
Stages  of  EDP  Growth",  Harvard  Business  Review,  Jan. -Feb. 
1974.  Their  concept  of  maturation  stages  is  supportive  of 
this  interpretation. 
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Thirdly,  it  is  striking  to  note  the  relative  positions  of  the 
four  different  types  of  information  systems:  Monitor  (J), 
Exception  (K),  Inquiry  (L),  and  Analysis  (M).*  The  most  struc- 
tured and  straightforward  of  the  four  system  types.  Monitor 
systems,  is  rated  as  the  most  successful  of  the  four.  In  fact, 
this  activity,  with  its  strong  "Good"  rating  of  5.01,  was  rated 
the  highest  of  all  26  criteria.  Exception  systems,  similarly 
straightforward  in  their  design  and  implementation,  are  also 
rated  relatively  well;  they  received  an  above  average  rating  of 
4.54.  Inquiry  systems  and  Analysis  systems,  however,  fall  into 
the  category  of  relative  weakness.  Both  are  rated  in  the  "In- 
adequate" range  at  3.94  and  3.57,  respectively. 

The  relatively  lower  success  ratings  for  creating  more  Inquiry 
and  Analysis  systems  are  due  in  part  to  the  different  develop- 
ment processes  which  they  require.^  If,  for  example,  the  devel- 
opment process  for  Monitor  and  Exception  systems  was  equally 
successful  for  these  more  flexible,  managerially-oriented  sys- 
tem types,  it  is  unlikely  that  we  would  see  such  significant 
differences  in  their  respective  success  ratings.  In  addition 
to  this  developmental  factor,  the  relatively  lower  success 
ratings  for  Inquiry  and  Analysis  systems  may  also  be  related  to 
a  bias  against  them  in  the  project  approval  and  prioritization 
process  (because  they  tend  to  be  qualitatively  justified).'' 
Moreover,  the  rapid  growth  in  user  demand  has  not  allowed  suf- 
ficient time  for  the  necessary  scale-up  and  training 
activities  that  provide  a  basic  capacity  to  develop  such  sys- 
tems. * 

The  relatively  low  success  ratings  for  these  systems,  linked 
with  the  increasingly  strong  demand  for  such  systems  by  user 
management,  suggest  that  they  are  likely  candidates  for  top 
priority  improvement  efforts  in  I/S. 


These  four  types  of  systems  are  more  fully  described  in  CISR 
Working  Paper  #55,  op.  cit. 

Keen,  P.G.W.  and  M.S.  Scott  Morton,  Decision  Support  Sys- 
tems: An  Organizational  Perspective,  Addison-Wesley, 
1978;  and  Alloway,  R.M.,  "The  Complete  Life  Cycle  of  CBIS 
Projects",  Institute  of  International  Business  Working 
Paper,  1976. 

Alloway,  R.M.  and  M.  Jonikas,  "The  Project  Selection  Bias 
Against  Management  Support  Systems",  CISR  forthcoming, 
discusses  this  and  other  concepts  m  relation  to  a  systems 
development  life  cycle.  Also  see  Keen,  P.G.W. ,  "Value 
Analysis:  Justifying  Decision  Support  Systems" ,  CISR  Work- 
ing Paper  #64,  M.I.T.,  Oct.  1980. 

Alloway,  R.M.  and  J.T.  Nolte,  "Planning  Skill  Development 
for  Systems  Analysts",  CISR  Working  Paper  #51,  M.I.T., 
March  1980. 
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DOES  THE  SUCCESS  PATTERN  HOLD  FOR  ALL  FIRMS? 


The  pattern  of  relative  strengths  and  weaknesses  that  we  have 
been  discussing  is  based  on  the  data  from  all  nineteen  firms 
collectively.  A  logical  question  is  whether  or  not  this  pat- 
tern holds  for  each  of  the  firms  individually  or  whether  it  is  a 
distortion,  resulting  from  averaging  together  nineteen  differ- 
ent firms.  In  fact,  as  illustrated  by  Figure  7 ,  the  pattern  of 
success  ratings  that  we  observed  previously  does  hold  for  each 
of  the  individual  firms  as  well. 

In  Figure  7,  the  variance  of  the  nineteen  firms'  individual 
success  ratings  about  the  average  is  shown  for  each  of  the  26 
individual  criteria.  The  solid  line  sloping  downward  is  the 
same  average  success  ratings,  in  the  same  sequence,  as  listed 
in  Figure  6  on  page  12.  The  vertical  bars  show  that  the  ratings 
from  the  nineteen  firms  for  each  criterion  do  have  some  vari- 
ation around  the  average  success  rating. 

It  is  striking  to  note,  however,  that  the  company  variation 
within  each  criterion  is  much  smaller  than  the  range  of  average 
success  ratings  across  the  spectrum  of  the  26  criteria.  In 
other  words,  the  pattern  that  we  observed  in  Figure  5  holds 
reasonably  well  for  each  of  the  nineteen  individual  firms. 
Although  there  is  some  variation  within  each  criterion,  this 
variation  appears  to  exist  in  the  context  of  a  central  theme, 
or  general  pattern  of  I/S  strengths  and  weaknesses. 

Using  the  four  success  groupings  identified  earlier,  we  can  see 
that  the  bottom  of  the  vertical  lines  for  traditional  technical 
activities  (average  rating  less  the  variance  by  firm)  is  still 
higher  than  the  top  of  most  of  the  vertical  lines  for  oper- 
ations (average  rating  plus  variance  by  firm),  and  exceeds  by 
even  greater  margins  the  highest  ratings  in  the  third  and 
fourth  groups,  I/S  management  and  managerial  systems  develop- 
ment. More  specifically,  the  lowest  rating  for  criterion  J 
(developing  more  Monitor  systems)  is  still  higher  than  the 
highest  rating  for  criterion  K  (developing  more  Exception  sys- 
tems) and  is  much  higher  than  the  managerial  support  systems, 
criterion  L  (developing  more  Inquiry  systems)  and  criterion  M 
(developing  more  Analysis  systems)  in  the  group  comprising 
managerial  support  systems  development. 

Essentially,  then.  Figure  7  demonstrates  that  none  of  the 
nineteen  companies  deviates  very  far  from  the  general  pattern 
which  we  have  identified  for  the  sample  as  a  whole.  This  gener- 
al adherence  to  the  pattern  of  success  ratings  is  an  important 
observation,  since  it  takes  place  despite  the  diversity  among 
these  firms  in  industry  classification,  organizational  size, 
and  size  and  structure  of  the  I/S  department. 
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INTERMEDIATE  SUMMARY 


A  brief  recapitulation  may  be  helpful  at  this  point.  In  order 
to  assess  the  success  of  the  I/S  function,  a  definition  of  suc- 
cess was  developed  which  encompassed  both  the  performance  and 
the  relative  importance  of  25  separate  criteria  representing 
the  breadth  and  diversity  of  I/S  responsibilities.  Using  this 
definition  of  I/S  success,  data  was  collected  from  1058  user 
and  I/S  managers  in  nineteen  different  industrial  firms.  The 
resulting  analysis  of  this  data  suggests  that,  in  fact,  there 
is  a  pervasive  problem  with  I/S  success.  The  average  overall 
success  rating  is  only  4.28,  statistically  significant  and 
managerially  different  from  a  "Good"  rating  of  "5"  on  the 
scale . 

In  addition,  we  have  found  that  user  managers'  success  ratings 
by  company  range  from  "inadequate"  (3.28)  to  "good"  (4.86), 
reflecting  the  differences  commonly  observed  between  I/S 
departments.  The  range  of  success  by  criterion  is  even 
greater,  from  3.27  for  criterion  D  (training  programs  for 
users)  to  5.01  for  criterion  J  (developing  more  Monitor 
systems).  Within  this  range  of  success,  a  general  pattern  of 
relative  strengths  and  weaknesses  across  the  26  success  crite- 
ria is  readily  observed.  Efficiency-oriented  criteria 
relating  to  traditional  technical  activities  and  I/S  oper- 
ations have  been  rated  as  more  successful  than  the 
effectiveness-oriented  criteria  associated  with  management  of 
the  I/S  function  and  development  of  managerial  support 
systems.  This  is  a  pattern  which  holds  not  only  for  the  nine- 
teen firms  collectively,  but,  with  minor  variations,  for  each 
individual  firm  as  well. 

It  is  definitely  helpful  to  define  and  measure  success  for  the 
I/S  function  as  a  whole,  especially  in  reference  to  relative 
importance  as  well  as  performance.  Historically  we  in  the  I/S 
function  have  measured  performance  without  attention  to  rela- 
tive importance.  Moreover,  we  have  made  it  difficult  to  make 
trade-off  decisions  by  measuring  single  criteria  without  ref- 
erence to  others  that  are  also  important.  But  perhaps  most 
perilously,  we  have  focused  our  attention  within  the  I/S 
department,  failing  to  emphasize  the  opinions  of  middle  and 
senior  level  user  managers.  The  User  Needs  Survey  methodology 
is  designed  to  compensate  for  these  typical  shortcomings.  With 
an  integrated  definition  of  success  and  the  combined  opinions 
of  user  and  I/S  managers  we  can  aspire  to  better  understand  the 
underlying  dynamics  or  causes  of  low  success  in  -the.  I/S  func- 
tion. 
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V/HAT  IS  THE  UNDERLYING  CAUSE  OF  LOW  SUCCESS? 


The  previous  section  discussed  the  definition  and  measurement 
of  I/S  success  and  the  pattern  of  relative  strengths  and  weak- 
nesses from  the  point  of  view  of  I/S  and  user  managers  collec- 
tively. In  this  section,  we  will  contrast  the  respective 
ratings  of  user  and  I/S  managers  to  determine  whether  there  are 
significant  differences  in  their  opinions  that  will  help  to 
explain  the  overall  low  success  rating. 

Disagreement  and  conflict  between  user  and  I/S  opinions 
regarding  success  by  criteria  is  a  potential  and  appealing 
explanation  for  the  problem  of  low  I/S  success.  It  is 
possible,  for  example,  that  user  managers  simply  do  not  under- 
stand I/S  functions  well  enough  to  evaluate  their  success. 
Hence,  user  ratings,  which  constitute  sixty  percent  of  the  sam- 
ple, might  unfairly  bias  the  result. 

Alternatively,  it  is  possible  that  user  managers  and  I/S  manag- 
ers simply  disagree  on  the  components  of  success:  importance 
or  performance.  Users  may  differ  from  I/S  regarding  the  impor- 
tance of  various  criteria,  for  example,  because  they  may  be 
unrealistic  and  naive  about  the  technical  prerequisites  for 
success.  Conversely,  differences  between  I/S  and  user  impor- 
tance ratings  may  result  from  I/S's  lack  of  understanding  or 
concern  with  users'  needs.  Moreover,  I/S  and  user  managers  may 
have  different  standards  of  "good"  performance  or  may  even  dis- 
agree on  the  actual  level  of  current  performance.  In  any  case, 
discovering  a  disparity  between  user  and  I/S  assessments  would 
help  us  considerably  in  understanding  the  underlying  dynamic 
causing  low  I/S  success. 


DO  USER  AND  I/S  MANAGERS  DISAGREE  ON  SUCCESS? 


The  consensus  of  popular  opinion,  as  well  as  the  frequent  judg- 
ment of  I/S  professionals,  holds  that  users  and  I/S  managers 
disagree  on  nearly  everything,  particularly  on  the  success  of 
I/S.  The  User  Needs  Survey  results  suggest,  in  contrast,  that 
there  is  a  remarkable  degree  of  similarity  in  the  two  groups' 
assessments  of  success  for  each  of  the  25  criteria. 

Figure  8  provides  a  graphic  representation  of  this  surprising, 
and  encouraging,  agreement  between  I/S  and  user  managers.  If 
user  and  I/S  success  ratings  for  the  25  criteria  were  exactly 
the  same,  the  letters  A  through  Z  (representing  each  of  the 
criteria  listed  in  Figure  3  on  page  5)  would  lie  in  a  straight 
diagonal  line  on  the  graph.  From  a  statistical  perspective, 
the  Spearman  Rho  correlation  coefficient  would  be  1 . 00 .  In 
fact,  the  correlation  coefficient  is  0.85  at  a  significance 
level  of  .0001  (indicating  that  there  is  one  chance  in  10,000 
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FIGURE  8.     COMPARISON  OF  I/S  AND  USER  RATINGS  OF  SUCCESS 
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that  the  coefficient  is  not  0.85).  Graphically,  the  plots  for 
the  26  criteria  are  rather  tightly  clustered  along  the 
diagonal.  Whereas  the  statistics  prove  the  point,  the  graph- 
ical representation  in  Figure  8  illustrates  that  there  is  a 
high  degree  of  agreement  between  user  and  I/S  managers  about 
the  relative  success  of  the  26  I/S  criteria. 

Further,  the  graphic  pattern  indicates  chat  user  and  I/S  manag- 
ers agree  that  the  success  for  all  the  26  criteria  is  in  the 
middle  range  of  "3"  to  "5"  on  the  rating  scale.  None  of  the 
success  criteria  fall  below  the  "inadequate"  level.  Further, 
the  two  groups  agree  on  those  criteria  v/ith  the  lowest  success: 
training  programs  for  users  (D),  developing  more  analysis  sys- 
tems (M),  involvement  of  senior  user  managers  in  I/S  policy 
formulation  (N),  and  the  new  system  request  backlog  (I).  They 
also  agree  that  none  of  the  26  criteria  enjoys  "excellent"  suc- 
cess. There  is  room  for  improvement  on  even  the  most 
successful  criteria.  Notice  that  only  a  few  (particularly  cri- 
terion J,  developing  more  Monitor  systems,  and  criterion  H, 
technical  competence  of  the  I/S  staff)  are  assessed  by  both 
groups  at  a  success  level  which  may  be  described  as  "Good"  on 
the  rating  scale.  This  pattern  confirms  the  earlier  finding 
that  there  is  a  problem  with  overall  I/S  success. 

This  excellent  agreement  between  the  two  groups  of  managers 
regarding  the  26  criteria  for  success  indicates  that  the  prob- 
lem of  low  success  cannot  be  found  simply  in  disagreements 
between  users  and  I/S  professionals.  While  this  discovery  does 
not  provide  insight  into  the  underlying  cause  of  low  success, 
the  determination  that  users  and  I/S  managers  share  similar 
assessments  of  overall  I/S  success  is,  in  itself,  an  important 
finding.  If  recognized  and  utilized  it  provides  an  invaluable 
basis  for  cooperative  improvement  efforts. 

The  implications  of  this  finding  are  provocative.  First,  user 
managers  can  and  do  discriminate  between  different  I/S  crite- 
ria. Even  though  user  managers  rate  overall  I/S  success  as 
mediocre,  this  does  not  mean  that  they  uniformly  rate  all 
aspects  of  I/S  equally.  Further,  user  managers  have  little 
difficulty  in  judging  success  even  on  technically-oriented 
criteria.  User  managers  may  not  personally  possess  these  tech- 
nical skills,  but  the  survey  results  demonstrate  that  they  are 
quite  capable  of  assessing  technical  capabilities.  In  fact, 
they  appear  to  be  as  capable  as  I/S  managers  in  judging  techni- 
cal competence  at  this  macro,  managerial  level. 

Second,  and  contrary  to  popular  belief,  I/S  managers  are  not 
out  of  touch  with  their  user  communities.  It  would  be  easy  for 
I/S  managers  to  overrate  their  own  success.  Fortunately,  this 
industry  sample  of  I/S  managers  is  actively  interested  in  their 
users'  needs  and  has  avoided  "the  I/S  isolation  trap". 

The  third  provocative  implication  of  Figure  8  concerns  pros- 
pects for  the  future.  The  agreement  we  found  in  these  nineteen 
firms  suggests  that  there  is  a  solid  basis  for  the  resolution 
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of  current  and  future  problems  between  I/S  and  users.  The 
shared  assessment  of  I/S  success,  not  only  in  the  aggregate, 
but  for  each  of  the  26  individual  criteria  as  well,  indicates 
that  a  common  definition  of  the  problem  exists.  A  necessary 
prerequisite  for  improvement  is  a  shared  understanding  of  what 
constitutues  success  and  an  explicit  agreement  on  where  the 
problems  are.  For  these  nineteen  firms  collectively,  and  prob- 
ably for  I/S  as  an  industry,  this  crucial  condition  and 
foundation  for  future  joint  planning  and  action  is  well  met.' 


DO  USER  AND  US  MANAGERS  DISAGREE  ON  IMPORTANCE? 


Although  user  and  I/S  managers  agree  on  the  relative  success  of 
each  of  the  25  criteria,  it  is  possible  that  there  are  major 
disagreements  in  their  ratings  of  relative  importance.  If  so, 
this  would  be  a  significant  finding,  in  that  it  would  provide  a 
better  understanding  of  the  overall  low  success  and  a  basis  for 
the  development  of  I/S  priorities.  For  example,  I/S  managers 
might  rate  only  technical  considerations  as  very  important. 
Conversely,  user  managers  may  rate  only  those  criteria  which 
are  associated  with  their  direct  interests  and  needs  as  very 
important. 

Figure  9  provides  a  graphic  representation  of  user  and  I/S 
importance  ratings  for  the  26  success  criteria.  Once  again,  the 
plots  of  user  and  I/S  ratings  for  each  criterion  fall  closely 
along  the  diagnonal  that  represents  perfect  agreement.^"  Again 
we  have  found  a  pattern  of  remarkable  agreement  between  user 
and  I/S  managers. 

There  is  no  real  disagreement  about  what  is  important  for  I/S. 
What  I/S  professionals  think  is  important  is  also  viewed  by 
user  managers  as  important;  and,  equally  impressive,  what  user 
managers  think  is  important  is  similarly  evaluated  by  I/S  man- 
agers. Once  again  we  have  an  unanticipated  and  positive  indi- 
cation that  user  and  I/S  managers  are  not  far  apart  in  their 
assessments  of  the  I/S  function. 

Whether  I/S  has  influenced  users  or  users  have  influenced  I/S 
is  immaterial.  The  fact  remains  that  user  and  I/S  managers  in 
these  nineteen  firms,  and  perhaps  throughout  the  industry, 
agree  on  what  is  and  is  not  important  for  the  I/S  function. 


Caution:  This  is  not  true  for  all  firms  individually.  In 
the  User  Needs  Survey,  several  I/S  departments  with  the 
lowest  success  did  not  show  this  prerequisite  for  improve- 
ment. 

The  Spearman  Rho  correlation  coefficient  in  this  case  is 
.93,  at  a  significance  level  of  .0001. 
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FIGURE  9.   COMI'ARISON  OF  I/S  AND  USER  RATINGS  OF  IMPORTANCE 
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DO  USER  AND  I  IS  MANAGERS  DISAGREE  ON  PERFORMANCE? 


We  have  explored  two  possible  explanations  for  low  ratings  of 
I/S  success  only  to  find  strong  agreement  between  user  and  I/S 
managers.  A  third  alternative  is  possible,  and  would  seem 
quite  likely.  Perhaps  I/S  managers  and  user  managers  disagree 
on  the  performance  component  of  success.  Such  a  finding  would 
indicate  that  either  they  disagree  on  how  well  I/S  actually  is 
performing,  or  they  disagree  on  what  level  of  actual  perform- 
ance is  acceptable. 

We  employ  the  now  familiar  graphic  technique  to  check  this 
hypothesis.  And  again,  as  shown  in  Figure  10,  we  find  the  same 
diagonal  clustering  of  the  25  criteria  which  indicates  excel- 
lent agreement  between  the  two  groups. ^^  Evidently  the  cause  of 
low  success  is  not  to  be  found  in  disagreements  between  users 
and  I/S  managers  on  either  the  actual  level  of  performance  or 
its  acceptability. 

Although  we  have  yet  to  find  any  clarification  of  the  low  suc- 
cess ratings  in  differences  of  opinion  between  users  and  I/S 
professionals,  we  have  again  discovered  some  encouraging 
results.  In  spite  of  their  different  organizational  positions 
and  interests,  users  and  I/S  managers  have  very  similar  views 
of  the  actual  performance  of  all  the  I/S  success  criteria. 
Although  one  criterion  is  rated  slightly  higher  in  performance 
by  user  managers  (criterion  E,  data  security  and  privacy),  and 
some  are  rated  marginally  higher  by  I/S  managers  (e.g.,  crite- 
rion C,  hardware  and  system  downtime),  all  of  the  performance 
ratings  fall  within  the  diagonal  band  illustrating  agreement. 

Success  criterion  C  (hardware  and  systems  downtime)  is,  in 
fact,  an  interesting  example  of  this  basic  agreement  between 
the  two  groups.  It  is  quite  possible  that  the  same  percentage 
of  downtime  would  be  rated  quite  differently  by  users  and  I/S 
managers.  A  rate  of  5%  downtime,  for  example,  might  be  consid- 
ered unsatisfactory  by  user  management  and,  at  the  same  time, 
be  considered  quite  good  by  I/S  management.  Instead,  both  man- 
agement groups  rate  the  performance  of  system  downtime 
approximately  the  same. 

The  surprising  implication  of  this  agreement  on  criterion  C,  as 
well  as  the  agreement  on  other  technically-oriented  criteria, 
is  that  user  managers  do  not  differ  significantly  from  I/S  pro- 
fessionals in  their  assessment  of  the  technical  performance  of 
the  I/S  function.  In  fact,  it  appears  that  they,  like  I/S  man- 
agers, have  a  realistic  understanding  of  what  constitutes  good 
performance  in  the  technical  arena  of  I/S  activities.  Con- 
versely, it  is  important  to  note  that  I/S  managers  appear  to  be 


The  Spearman  Rho  correlation  coefficient  for  the  two  sets 
of  performance  ratings  is  .85,  at  the  .0001  significance 
level . 
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FIGURE  10.  COMPARISON  OF  I/S  AND  USER  RATINGS  OF  PERFORMANCE 
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as  sensitive  as  user  managers  to  the  performance  issues  associ- 
ated with  non- technical,  user-oriented  criteria,  such  as,  the 
attitude  of  I/S  personnel  toward  users  (G),  I/S  support  for 
users  in  preparing  proposals  for  new  systems  (U)  and  training 
programs  for  users  (D)  . 

The  fact  that  all  of  these  criteria,  which  represent  a  diverse 
range  of  I/S  issues,  are  rated  similarly  by  user  and  I/S  manag- 
ers suggests  that  certain  stereotypes  about  the  two  groups  may 
not  be  entirely  valid.  Users,  for  example,  are  usually  viewed 
as  unsophisticated  in  I/S  technical  matters.  However,  their 
judgment  of  technical  I/S  performance  closely  matches  that  of 
the  I/S  professionals  themselves.  On  the  other  hand,  I/S  per- 
sonnel are  traditionally  considered  to  be  insensitive  to  user 
needs  and  requirements.  Yet,  they  demonstrate  an  assessment  of 
performance  for  user-related  criteria  that  is  remarkably  close 
to  that  of  user  managers  themselves. 


INTERMEDIATE  SUMMARY 


Although  we  have  still  to  discover  the  underlying  dynamic  which 
accounts  for  the  relatively  low  success  ratings  of  the  I/S 
function,  we  continue  to  find  heartening,  encouraging  patterns 
suggesting  major  areas  of  agreement  between  user  and  I/S  manag- 
ers as  they  assess  the  success,  importance,  and  performance  of 
the  26  criteria. 

The  question  remains,  however.  What  is  the  underlying  "cause" 
of  low  I/S  success?  We  have  found  that  user  managers  and  I/S 
managers  rate  overall  I/S  success  as  marginal.  Further,  based 
on  the  success  ratings  for  the  individual  criteria,  we  found  a 
general  pattern  which  separates  them  into  four  groupings:  tra- 
ditional technical,  operations  and  control,  I/S  management, 
and  managerial  support  systems  development.  In  searching  for 
some  underlying  dynamic  that  would  explain  these  success 
ratings,  we  have  explored  three  alternatives,  each  one  focus- 
ing on  possible  differences  of  opinion  between  user  and  I/S 
managers.  We  have  compared  user  and  I/S  manager  ratings  of 
success,  importance,  and  performance,  and  we  have  found 
remarkable  agreement.  The  implication  of  this  is  that  we  must 
look  elsewhere  if  we  are  to  find  the  underlying  cause  of  low  I/S 
success.  It  cannot  be  explained  as  a  difference  of  opinion 
between  I/S  and  user  managers. 
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FOUND:     THE  "CAUSE"  OF  LOW  SUCCESS 


To  no  avail,  we  have  analyzed  each  of  the  possible  differences 
between  user  and  I/S  opinions,  hoping  to  find  a  significant 
difference  that  could  cause  the  low  success  ratings  for  indi- 
vidual criterion  and  the  I/S  function  as  a  whole.  Contrary  to 
popular  opinion,  however,  the  cause  is  not  to  be  found  in  I/S 
versus  user  differences  of  opinion.  Pursuing  a  completely  dif- 
ferent tack,  we  compared  users'  ratings  of  importance  with 
users'  ratings  of  performance.  This  comparison  is  the  key  to 
our  understanding  the  cause  of  low  I/S  success. 

Figure  11  illustrates  this  diagnostic  technique.  It  is  essen- 
tially the  same  technique  we  have  used  earlier,  except  that  now 
we  are  comparing  users'  ratings  of  importance  with  users' 
ratings  of  performance.  As  in  previous  graphs,  good  agreement 
would  result  in  a  diagonal  pattern.  But  the  image  of  Figure  11 
is  strikingly  different  from  that  of  previous  graphs.  Rather 
than  falling  along  the  diagonal,  the  plots  of  the  26  criteria 
are  randomly  scattered.  In  contrast  to  the  excellent  corre- 
lation statistics  for  previous  graphs,  the  correlation 
coefficient  for  Figure  11  is  almost  zero  (.08)  and  is  not  even 
close  to  being  statistically  significant  (.70).  Clearly  we 
have  found  a  dramatic  departure  from  the  previous  patterns  of 
close  agreement.^* 

This  random  scatter  of  importance  and  performance  ratings  dis- 
closes that  user  managers  see  no  relationship  between  the 
importance  of  a  particular  criterion  and  its  corresponding 
performance.  For  example,  two  criteria  that  are  rated  at  wide- 
ly different  levels  of  performance,  low-rated  M  (developing 
more  Analysis  systems)  and  high-rated  E  (data  security  and  pri- 
vacy) have  almost  the  same  importance  ratings.  Conversely,  two 
criteria  which  users  judge  to  be  equal  m  performance,  criteria 
Z  (I/S  profitability)  and  W  (availability  and  timeliness  of 
report  delivery)  are  seen  by  users  as  being  very  different  in 
importance.  This  relationship  is  not  only  statistically  ran- 
dom, but  more  critically,  managerially  unacceptable . 

From  the  users'  perspective,  then,  it  appears  that  whatever 
performance  I/S  has  achieved  has  not,  unfortunately,  been 
related  to  importance.  Simply  put,  user  managers  do  not  see  a 


It  should  be  noted  that  the  range  of  the  scale  in  Figure  11 
has  been  changed  from  the  original  "1"  to  "7"  range,  to  a 
reduced  range  of  "3"  to  "7".  This  change  has  been  made 
solely  to  illustrate  the  randomness  more  effectively.  The 
correlation  statistic  verifies  in  two  ways  this  randomness. 
First  the  low  Spearman  Rho  correlation  coefficient  (.08) 
indicates  a  lack  of  any  meaningful  relationship.  Second, 
the  high  value  of  the  significance  level  (.70)  indicates 
little  confidence  in  any  kind  of  relationship. 
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FIGURE    11.      COMPARISON   OF   USER  RATINGS   OF    IMPORTANCE  AND   PERFORMANCE 
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pattern  that  relates  performance  to  importance.  It  is  not  that 
all  high  performing  criteria  are  unimportant;  rather,  the  jus- 
tifiable expectation  that  they  should  consistantly  tend  to  be 
important  is  not  fulfilled.  At  least  from  the  vantage  point  of 
user  managers,  there  appears  to  be  no  relationship  between  the 
two  considerations  for  most  criteria. 

As  further  illustration  of  this  lack  cf  relationship  between 
users'  ratings  of  importance  and  performance  consider  criteria 
J,  K,  L,  and  M  which  are  the  four  different  types  of  information 
systems:  Monitor,  Exception,  Inquiry,  and  Analysis.  When  the 
importance/performance  pairings  are  compared  for  the  four  dif- 
ferent systems  types,  there  actually  appears  to  be  an  inverse 
relationship  between  the  two  factors  --  the  more  important  the 
system  type,  the  lower  its  rating  of  performance.  Developing 
more  Monitor  systems  (J)  is  judged  by  users  to  be  least  impor- 
tant of  the  four.  Yet  users  clearly  evaluate  I/S  as  having 
higher  performance  on  criteria  J.  Developing  more  Exception 
systems  (K)  is  considered  to  be  somewhat  more  important  than 
criterion  J.  However,  I/S  is  evaluated  by  users  as  having  per- 
formed less  well  on  criterion  K  than  on  J.  And  for  the  two  most 
important  system  types.  Inquiry  and  Analysis  systems  (criteria 
L  and  M,  respectively),  I/S  performance  is  judged  by  users  as 
the  lowest  for  all  systems  types. 

The  results  of  this  analysis  suggest  that  we  have  uncovered  the 
underlying  cause  which  is  reponsible,  at  least  in  the  opinion 
of  user  managers,  for  the  phenomenon  of  low  success.  User  man- 
agers see  no  relationship  between  the  importance  and  perform- 
ance of  most  success  criteria.  This  is  an  I/S  strategic 
planning  and  resource  allocation  problem.  The  I/S  function  has 
not  set  its  goals  and  allocated  its  resources  such  that  those 
criteria  of  greatest  importance  have  acceptable  levels  of  per- 
formance. I/S  management  may  have  been  trying  to  achieve  a 
balance  between  importance  and  performance,  but  to  date  it  has 
not  been  accomplished. 

But  do  I/S  managers  see  the  same  cause  for  low  success?  Or  do 
their  ratings  indicate  a  closer  alignment  of  performance  and 
importance  for  the  25  success  criteria?  Figure  12  provides  the 
answer.  Again  we  see  an  undeniably  random  dispersion  of  impor- 
tance/performance criteria  on  the  graph. ^'  I/S  managers  agree 
with  users;  they  do  not  see  any  justifiable  relationship 
between  the  two  dimensions  of  success.  What  is  important  may 
or  may  not  have  high  performance,  and  criteria  with  high  per- 
formance may  or  may  not  be  relatively  important.  For  example, 
I/S  managers  see  the  same  inverse  relationship  between  impor- 
tance and  performance  that  user  managers  perceived  for  the  four 
major  system  types.   The  less  important  system  types.  Monitor 


The  Spearman  Rho  correlation  statistics  (coefficient  of 
.10,  at  a  significance  level  of  .52)  again  demonstrate  the 
randomness  of  this  plot. 
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FIGURE  12.   COMPARISON  OF  I/S  RATINGS  OF  IMPORTANCE  AND  PERFORMANCE 
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(J)  and  Exception  (K),  have  higher  performance  than  the  more 
important  system  types.  Inquiry  (L)  and  Analysis  (M)  . 

It  appears  from  the  results  of  Figure  11  and  Figure  12  that  we 
have  finally  uncovered  the  basic  dynamic  which  accounts  for 
both  I/S  and  users'  low  ratings  cf  I/S  success.  Neither  group 
of  managers  sees  any  relationship  between  the  two  critical  com- 
ponents of  success,  the  importance  and  the  performance  of  a 
particular  criterion.  It  is  as  though  one  of  the  basic  tenets 
of  management,  allocating  resources  to  align  performance  with 
importance,  is  being  violated.  Further,  it  is  striking  that 
both  management  groups  perceive  the  situation  in  the  same  way. 

This  underlying  imbalance  betv/een  importance  and  performance 
ratings  raises  serious  questions  about  the  I/S  planning  proc- 
ess for  the  firms  participating  in  the  User  Needs  Survey  and, 
presumably,  for  other  firms  as  well.  The  results  suggest  that 
the  I/S  function  has  not  succeeded  in  the  past,  for  whatever 
reason,  in  improving  its  performance  on  those  things  that  are 
judged  to  be  important  by  both  I/S  and  user  managers.  Although 
it  is  likely  that  many  I/S  managers  have  not  yet  recognized 
importance-performance  discrepancies  as  the  cause  of  low  suc- 
cess, it  is  also  true  that  many  I/S  managers  already  have  been 
trying  to  change  the  importance-performance  gaps  they  do 
perceive.  Despite  the  best  efforts  of  individual  managers, 
policy,  procedure  and  priority  changes  will  not  occur  until 
there  is  a  broad-based  understanding  of  the  problem.  Our 
findings  indicate  that  this  imbalance  between  importance  and 
performance  will  continue  until  a  process  is  established  which 
provides  a  basis  for  communication  and  an  explicit  methodology 
for  problem  diagnosis  and  priority  setting. 
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PRIORITIES  FOR  l/S 


The  underlying  cause  of  low  success,  as  identified  in  the  pre- 
vious section,  is  lack  of  a  consistent  relationship  between 
importance  and  performance  for  the  25  criteria  of  I/S  success. 
We  have  seen  that  high  ratings  of  importance  are  often  associ- 
ated with  low  ratings  of  performance.  Conversely  low  ratings 
of  importance  are  often  associated  with  relatively  high 
ratings  of  performance.  In  effect  we  are  observing  a  gap 
between  importance  and  performance  ratings.  Instead  of  being 
approximately  equal,  one  rating  is  often  larger  than  the  other. 


IMPORTANCE/PERFORMANCE  CAPS  AND  PRIORITIES 


A  gap  between  the  two  ratings  of  importance  and  performance  is 
not  found  for  all  26  criteria,  though  it  does  appear  in  the 
great  majority  of  paired  ratings.  In  some  cases  the  importance 
rating  is  relatively  higher  than  the  performance  rating,  while 
in  others  the  opposite  is  true.  By  understanding  the  charac- 
teristics of  these  gaps  between  importance  and  performance 
ratings,  we  can  begin  to  identify  the  key  areas  for  I/S 
improvement,  in  other  words,  the  top  priorities  for  the  I/S 
function. 

Figure  13  provides  a  graphic  representation  of  the  gaps 
between  importance  and  performance  ratings  for  each  of  the  26 
success  criteria.  For  each  criterion,  the  rating  of  perform- 
ance has  been  subtracted  from  the  rating  of  importance.  If  the 
resulting  gap  is  positive,  it  indicates  that  the  importance 
rating  for  that  particular  criterion  exceeded  its  rating  of 
performance.  If  the  gap  is  negative,  it  indicates  that  the 
importance  rating  was  actually  less  than  the  associated  per- 
formance rating.  This  computation  of  importance-performance 
gaps  has  been  done  for  both  groups  of  managers,  users  and  I/S 
alike,  with  the  resulting  gaps  shown  on  the  chart  represented 
in  Figure  13 . 

Once  again  we  see  the  familiar  diagonal  clustering  pattern, 
indicating  agreement  between  user  managers  and  I/S  managers. 
This  time,  however,  the  agreement  is  not  on  ratings  of  impor- 
tance alone  or  performance  alone.  Rather,  the  agreement  illus- 
trated in  Figure  13  concerns  the  difference  between  the  two 
ratings:  the  importance-performance  gap. 

Using  these  importance-performance  gaps,  we  can  now  determine 
the  top  priorities  for  I/S.  Those  criteria  which  have  the 
largest  gap  between  importance  and  performance  ought  to  be  the 
primary  focus  of  our  efforts  to  improve  I/S  success.  These 
criteria,  which  appear  in  the  upper  right  corner  of  the  graph 
are  held  by  I/S  and  user  managers  alike  to  be  of  high  importance 
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FIGURE  13.   IMPORTANCE  -  PERFORMANCE  GAPS  FOR  I/S  AND  USERS 
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and  to  have  experienced  relatively  low  performance.  These  are 
the  criteria  which  should  receive  the  most  attention  and 
resources  in  order  to  raise  the  overall  success  ratings  of  the 
I/S  function. 

The  criteria  whose  ratings  of  performance  exceed  their  ratings 
of  importance  appear  in  the  lower-left  corner  of  the  graph. 
They  have  negative  importance-performance  gaps.  These  are  the 
criteria  which  I/S  and  user  managers  agree  are  relatively  low 
in  importance,  but  which  they  also  rate  as  relatively  high  in 
performance.  It  is  important  to  note,  however,  that  the  per- 
formance ratings  for  these  criteria  are  high  only  in  the 
relative  sense  as  none  of  the  criteria  (as  we  saw  in  Figure  10 
on  page  24)  have  a  rating  much  higher  than  "good"  or  "5"  on  the 
seven-point  scale. 

The  criteria  appearing  in  the  lower-left  corner  of  the  chart 
should  not  be  considered  as  priorities,  given  limited  I/S 
resources.  Even  though  there  is  room  for  improvement,  the  rel- 
atively low  importance  ratings  for  these  criteria  would  sug- 
gest that  these  areas  are  experiencing  something  of  an 
"over-performance"  in  relation  to  their  importance.  To  fur- 
ther concentrate  on  improving  performance  in  these  areas  would 
be  a  waste  of  scarce  management  time  and  I/S  resources. 

For  example,  criterion  J  (developing  more  Monitor  systems)  has 
a  relatively  high  performance  rating;  however,  additional 
resources  and  attention  could  be  devoted  to  raise  it  even  high- 
er, perhaps  even  to  the  truly  excellent  range  of  a  6  or  7.  The 
wisdom  of  such  action  would,  however,  be  highly  suspect  since 
the  importance  of  developing  more  monitoring  systems  was  rated 
at  a  relatively  low  level.  Hence,  this  criterion  and  others 
like  it  should  not  be  considered  as  high  I/S  priorities  for 
improvement.  Instead,  it  would  seem  more  appropriate  to  opt 
for  a  "holding  action"  which  would  maintain  their  actual  level 
of  performance.  Obviously,  importance  and  performance  ratings 
are  all  relative  and  I/S  management  should  carefully  consider 
the  consequences  before  reducing  the  actual  level  of  perform- 
ance on  any  criterion. 


DETERMINING  LEVELS  OF  PRIORITY 

The  high  priority  criteria  in  the  upper-right  corner  of 
Figure  13  require  closer  examination,  in  order  to  identify 
levels  of  priority  that  can  be  used  in  developing  an  action 
plan  for  increased  I/S  success.  On  Figure  13,  semi-circles 
have  been  drawn  rather  arbitrarily  to  define  four  levels  of 
priority  for  the  fourteen  criteria  with  the  largest 
importance-performance  gaps. 

Figure  14  summarizes  this  grouping  of  the  criteria  into  four 
priority   levels.    It   is   important,   at   this   point,   to 
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FIGURE  14.  PRIORITIES  FOR  ACTION 


TOP  PRIORITY 

0.   RESPONSIVENESS  TO  USER  NEEDS 

SECOND  PRIORITY 

M.   DEVELOPING  MORE  ANALYSIS  SYSTEMS 

A.   COMMUNICATION  WITH  MANAGERIAL  USERS 

D.   TRAINING  PROGRAMS  FOR  USERS  IN  GENERAL  I/S  CAPABILITIES 

T.   USER  ORIENTED  SYSTEMS  ANALYSTS  WHO  KNOW  USER  OPERATIONS 

THIRD  PRIORITY 

P.   I/S  STRATEGIC  PLANNING  AND  ALLOCATION  OF  RESOURCES  TO 

KEY  BUSINESS  AREAS 
S.   IMPROVING  NEW  SYSTEM  DEVELOPMENT  (TIME,  COST,  QUALITY, 

DISRUPTIONS) 

FOURTH  PRIORITY 

L.   DEVELOPING  MORE  INQUIRY  SYSTEMS 

G.   THE  ATTITUDE  OF  I/S  PERSONNEL  TOWARD  USERS 

N.    INVOLVEMENT  OF  SENIOR  USER  MANAGERS  IN  I/S  POLICY 

FORMULATION  AND  EVALUATION 
Y.   REPORT  CONTENTS  (RELEVANCE,  CURRENTNESS,  FLEXIBILITY, 

ACCURACY) 
F.   QUALITY  OF  I/S  SYSTEMS  ANALYSTS 

W.   AVAILABILITY  AND  TIMELINESS  OF  REPORT  DELIVERY  TO  USERS 
C.   HARDWARE  AND  SYSTEMS  DOWNTIME 
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re-emphasize  that  this  set  of  priorities  is  not  a  theoretical 
construct.  Rather,  it  has  been  derived  from  the  opinions  of  an 
industry-wide  sampling  of  a  thousand  user  and  I/S  managers. 
Further,  as  we  have  seen  in  previous  sections,  the  opinions  of 
these  managers  have  been  sophisticated  and  discriminating. 
What  we  have  uncovered  through  our  analysis  of  the  differences 
between  the  ratings  of  importance  and  performance,  the 
importance-performance  gaps,  is,  therefore,  a  widespread  and 
consistent  set  of  priorities  for  improving  I/S  success. 

The  first  item  on  this  agenda  for  improved  I/S  success  is  cri- 
terion 0,  responsiveness  to  user  needs.  This  criterion's  posi- 
tion of  foremost  priority  is  not  a  trivial  restatement  of  the 
obvious.  Instead,  it  may  well  be  an  indication  of  the  stage  of 
I/S  development  for  these  nineteen  firms.  Most  of  them  can  gen- 
erally be  characterized  in  the  Nolan  framework  as  having  made 
the  transition  from  the  third  stage,  with  its  emphasis  on  con- 
trol, efficiency,  and  cost  containment,  to  the  fourth  stage  of 
maturity,  with  its  new  emphasis  on  responsiveness  to  user 
needs .  ^^ 

Successfully  achieving  responsiveness  to  user  needs  will  nec- 
essarily include  the  related  activities  represented  by  the 
second,  third  and  fourth  levels  of  priority  in  Figure  14.  For 
example,  increased  responsiveness  to  user  needs  will  require 
the  development  of  more  Analysis  systems  (criterion  M)  and  more 
Inquiry  systems  (L).  The  development  of  these  systems  types, 
however,  will  depend  in  turn  on  progress  on  other  top  priority 
criteria,  such  as  training  programs  for  users  (D)  and 
user-oriented  systems  analysts  who  know  user  operations  (T) . 
In  addition,  criteria  associated  with  user  involvement  in  I/S 
planning,  such  as  communication  with  managerial  users  (A),  I/S 
strategic  planning  and  allocation  of  resources  to  key  business 
areas  (P),  and  involvement  of  senior  user  managers  in  I/S  poli- 
cy formulation  (N),  will  also  be  critical  to  the  first  priority 
of  responsiveness  to  user  needs,  and  therefore  to  a  successful 
fourth-stage  I/S  function. 


IMPROVEMENT  EFFORTS  FOR  INCREASED  I/S SUCCESS 


The  recommended  top  priorities  for  improvement  of  I/S  success 
are  unequivocal  and  hence  of  considerable  help  in  I/S  goal  set- 
ting and  resource  allocation.  However,  simultaneously  under- 
taking improvement  efforts  on  these  fourteen  high  priority 
criteria  is  at  least  formidable,  and  perhaps  overwhelming.  It 
will  probably  take  several  years  before  a  plan  of  action  for 
each  is  developed  and  implemented. 


^^  Gibson  and  Nolan,  op.cit. 
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FIGURE   15, 
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In  addition  to  determining  levels  of  priority  as  a  basis  for 
I/S  planning,  it  is  also  possible  to  discern  certain  themes 
that  emerge  across  the  top  priority  criteria.  These  themes,  as 
illustrated  in  Figure  15,  are  really  different  categories  of 
improvement  efforts.  For  example,  certain  of  the  top  priority 
criteria  can  be  viewed  as  Enabling  efforts,  while  others  are 
more  discrete  Directed  Actions.  Both  of  these  types  of 
improvement  efforts  contribute  to  the  more  subjective,  yet 
extremely  significant.  Outcomes. 

Thus,  the  top  fourteen  priority  criteria  can  be  grouped  in  a 
useful  manner  to  facilitate  developing  I/S  plans.  First, 
several  criteria  relate  to  process  changes  and  the  building  of 
an  improved  infrastructure.  They  are  "enabling"  conditions 
which,  when  present,  make  it  easier  to  succeed  on  the  other 
more  task-oriented  criteria.  Education  programs  and  changes 
in  managerial  procedures  fall  into  this  Enabling  category. 
These  efforts  require  an  investment  of  time  and  resources  for 
future  successes  in  other  areas.  As  such,  they  require  a  long- 
er term  perspective  and  will  yield  invisible,  dispersed,  but 
significant  benefits. 

The  Enabling  efforts  can  be  interpreted  to  consist  of  two 
kinds.  The  first  covers  criteria  which  involve  process 
changes:  I/S  strategic  planning  (P),  improving  the  new  systems 
development  process  (S),  and  involvement  of  senior  user  manag- 
ers in  I/S  policy  formulation  (N) .  Each  of  these  criteria  is 
associated  with  developing  new  processes  for  planning  and 
developing  systems.  They  have  in  common  a  concern  for  changing 
approaches  and  methods,  rather  than  attending  to  specific 
projects  or  actions.  These  process  changes  would  provide  a 
foundation  or  a  capacity  for  future  growth  and  development. 

Education  programs  constitute  the  second  type  of  Enabling 
activity.  Each  of  these  three  criteria  highlights  the  need  for 
cross-functional  education  and  training  of  user  and  systems 
personnel  as  a  basis  for  future  I/S  improvement.  User-oriented 
systems  analysts  who  know  user  operations  (T)  and  the  quality 
of  systems  analysts  (F)  are  priority  criteria  which  require 
concentrated  training  and  educational  efforts  in  order  to 
increase  the  underlying  capacity  of  I/S  to  perform  its  function 
successfully.  Similarly,  training  programs  for  users  in  gen- 
eral I/S  capabilities  (D)  provide  a  basis  for  their 
understanding  of  the  potentials  and  limitations  of  the  I/S 
function.  This  is  a  user  capability  which  is  critical  for  the 
successful  realization  of  such  Directed  Action  criteria  as 
developing  more  Analysis  systems  (M)  and  of  such  Outcome  crite- 
ria as  communication  with  managerial  users  (A)  .  Each  of  these 
education-related  Enabling  efforts,  therefore,  should  be  con- 
sidered when  planning  improvement  efforts  for  increased  I/S 
success. 

The  ability  to  expend  organizational  resources  on  such 
long-term  benefits  necessitates  a  credibility  which  is  based 
upon  current  success.   For  this  reason,  it  is  important  to 
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undertake  Directed  Actions  which  can  provide  a  "protective 
umbrella"  for  the  longer-term  Enabling  efforts. 

The  Directed  Actions  category  includes  priority  criteria  that 
involve  a  variety  of  specific  actions,  each  of  which  can  be 
structured  as  a  discrete  project.  These  task-oriented 
projects  are  visible,  concrete,  and  should  have  a  more  immedi- 
ate payback.  Two  of  these  efforts  are  associated  with  develop- 
ing particular  system  types:  Analysis  systems  (M)  and  Inquiry 
systems  (L).  Both  of  these  directed  actions  would,  inevitably, 
contribute  to  the  overall  outcome  of  responsiveness  to  users 
needs  (0)  and  should  therefore  be  considered  as  critical  compo- 
nents of  an  I/S  strategic  plan. 

In  addition,  there  are  other  Directed  Actions  which  will  con- 
tribute over  time  to  greater  success  on  the  Outcome  criteria. 
Specific  projects  that  increase  the  relevance,  timeliness, 
flexibility,  and  accuracy  of  reports  (Y,  W)  will  undoubtedly 
contribute  to  improved  success  for  such  Outcome  criteria  as 
responsiveness  to  user  needs  (0)  and  attitudes  of  users  to  I/S 
(G).  Reducing  hardware  and  system  downtime  (C),  especially  as 
more  on-line  Inquiry  and  Analysis  systems  are  installed,  will 
support  these  Outcome  criteria  as  well. 

Finally,  several  top  priority  criteria  are  less  tangible  as 
they  involve  attitudes  and  communication  and  are  therefore 
much  more  difficult  to  address  directly.  These  "outcome-like" 
criteria  are  not  susceptible  to  brute  force  resource  allo- 
cation. Nonetheless,  they  can  be  improved  through  an  explicit 
recognition  by  I/S  managers  that  these  issues  constitute  top 
priority  aspects  of  their  own  job  descriptions. 

Fortunately,  none  of  these  types  of  criteria  are  in  conflict 
with  one  another.  As  Enabling  conditions  improve,  it  will  be 
easier  to  succeed  in  Directed  Action  and  Outcome-like 
criteria.  As  Directed  Action  projects  are  successfully  imple- 
mented. Outcome-like  attitudes  and  cooperation  on  Enabling 
activities  will  improve.  As  Outcome-like  attitudes  improve, 
it  will  be  easier  to  invest  in  Enabling  conditions  and  achieve 
effective  cooperation  on  Directed  Action  projects.  Because  of 
these  interrelationships,  it  is  important  to  review  each  area 
carefully,  and  begin  planning  for  the  specific  activities  and 
efforts  which  will  facilitate  their  implementation. 
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TOP  PRIORITIES:    A  MANDATE  FOR  l/S  ACTION 


The  top  priorities  that  we  have  just  identified  provide  us  with 
a  mandate  for  improving  the  overall  success  of  the  I/S 
function.  In  effect,  this  set  of  priorities  and  improvement 
efforts  represents  the  collective  judgement  of  key  I/S  profes- 
sionals and  user  managers,  in  typical  private  sector  settings, 
about  what  needs  to  be  addressed  to  improve  I/S  success. 

And,  as  we  have  seen  continuously  throughout  this  paper,  the 
collective  judgment  of  I/S  and  user  managers  is  remarkably  con- 
sistent. Not  only  do  the  two  groups  agree  on  ratings  of  I/S 
succcess  but  they  also  share  similar  assessments  of  the  two 
basic  components  of  success  --  importance  and  performance.  And 
perhaps  most  notable,  they  both  perceive  the  same  underlying 
cause  of  low  success  --  the  lack  of  a  managerially  justifiable 
relationship  between  importance  and  performance.  With  such 
similar  perceptions  of  importance-performance  gaps,  the  top 
priorities  for  improving  I/S  success  are  remarkably  clear. 
They  represent,  in  effect,  a  bipartisan  plan  for  improving 
overall  I/S  success. 

Given  the  strong  agreement  between  I/S  professionals  and  user 
managers,  it  appears  that  our  focus  should  now  be  on  action: 
both  short-term  directed  actions  like  developing  more  inquiry 
and  analysis  systems;  and  also  longer-term,  investment- 
oriented  enabling  efforts  like  training  programs,  improving 
the  new  systems  development  process,  and  implementing  better 
methods  for  I/S  planning  and  resource  allocation.  These 
efforts  must  begin  together,  however,  for  the  directed  actions 
should  provide  the  short-term  success  and  credibility  that 
will  allow  for  the  longer-term  investment  of  the  enabling 
efforts. 

None  of  these  actions  is  overwhelmingly  difficult.  Although 
the  specific  techniques  and  methodologies  may  still  be  evolv- 
ing, our  difficulty  in  accomplishing  such  actions  may  lie  more 
in  our  inability  to  focus  our  efforts  than  in  our  lack  of  expe- 
rience or  expertise.  The  techniques  and  methods  can  be 
learned,  can  be  experimented  with,  and  will  continue  to 
develop.  What  is  important  is  that  we  begin  the  effort,  that  we 
focus  our  discretionary  efforts  and  resources  on  those  specif- 
ic criteria  which  I/S  and  user  managers  agree  are  of  top 
priority. 

It  is  important  to  acknowledge,  however,  that  we  can  never 
expect  to  achieve  uniformly  high  success  ratings  across  all  26 
criteria.  There  will  always  be  areas  of  I/S  activity  which 
will  experience  relatively  low  success  (by  definition  the  rel- 
ative success  on  some  criteria  must  be  below  average).  There- 
fore, the  I/S  function  will  always  have  areas  with  relative 
need  for  improvement. 
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In  addition,  we  can  anticipate  that  the  rating  scale  will  con- 
tinue to  experience  an  inflation  of  expectations.  As  the  I/S 
function  increases  its  success,  more  will  be  expected  of  it. 
This  phenomenon  is  certainly  familiar  to  I/S  managers  who, 
because  of  past  successes,  are  subject  to  the  continued  expec- 
tation for  more  and  greater  success  (e.g..  What  have  you  done 
for  us  lately?).  The  result  is  that  the  entire  rating  scale  is 
continuously  adjusted  upwards  in  the  minds  of  both  user  and  I/S 
managers. 

In  spite  of  these  two  reasons  which  will  insure  that  the  I/S 
function  never  achieves  perfect  success,  the  techniques  and 
findings  that  we  have  established  in  this  research  suggest  that 
there  is  a  way  to  maximize  relative  overall  success.  By  defin- 
ing I/S  success  as  a  set  of  relevant  criteria  comprised  of  both 
importance  and  performance  components,  and  by  measuring  suc- 
cess from  both  user  and  I/S  perspectives,  we  have  developed  a 
method  which  allows  us  to  identify  critical  gaps  in  importance 
and  performance  ratings  and,  therefore,  to  determine  the  top 
priorities  for  I/S  action.  These  top  priorities,  and  the 
resulting  improvement  efforts,  provide  the  guidance  and  assur- 
ance that  we  maximize  the  overall  contribution  to  I/S  success 
by  expending  incremental  attention  and  resources  on  those 
areas  which  need  it  most. 

We  can  realistically  expect  to  maximize  our  present  contrib- 
ution to  overall  I/S  success  by  reducing  the  largest 
importance-performance  gaps.  We  must  also  expect  that 
improved  success  will  contribute  to  the  need  to  understand  and 
address  new  priorities  reflecting  new  importance-performance 
gaps.  However,  by  periodically  assessing  I/S  success  and 
determining  new  top  priorities  we  can  assure  that  we  remain 
sensitive  to  the  changing  requirements  in  the  complex, 
fast-paced  I/S  environment.  Moreover,  periodic  measurement  of 
I/S  success  will  provide  follow-up  evaluation  of  previously 
implemented  improvement  efforts.  The  User  Needs  Survey  defi- 
nition of  I/S  success  and  priorities  can  be  used  in  a 
management  planning  process  to  yield  a  strategic  plan  for 
goal-setting  and  resource  allocation  which  insures  the  maximum 
contribution  to  overall  I/S  success. 
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